Improved Software Testing for Open Architecture

Proceedings Paper (for Acquisition Research Program)Applying traditional manual US Navy testing practices to OA systems will limit many benefits of OA, such as system scalability, rapid configuration changes, and effective component reuse. Pairing profile-driven automated software testing with test reduction techniques should enable these benefits and keep resource requirements at feasible levels. Test cases generated by operational profiles have been shown to be more effective than those developed by other methods, such as random or selective testing, and more resource-efficient than exhaustive approaches. This research effort increases the fidelity of the operational profile, creating an environment model referred to as a High-Fidelity Profile Model (HFPM) that can statistically describe individual software inputs. Samples from the HFPM''s probability distributions can generate operationally realistic or overly-stressful test cases for software modules under test. This process can be automated and paired with output checking functions, enabling automated effective software testing, and potentially improving reliability. Such models would be ideal for US Navy Open Architecture (OA) software because of the defined interface standards. HFPMs can enable effective testing in software reuse applications and are ideal for testing multiple releases of maturing software. This research defines the HFPM, presents a methodology to develop, validate, and apply it.Naval Postgraduate School Acquisition Research ProgramApproved for public release; distribution is unlimited

Putting Teeth into Open Architectures: Infrastructure for Reducing the Need for Retesting

Proceedings Paper (for Acquisition Research Program)The Navy is currently implementing the open-architecture framework for developing joint interoperable systems that adapt and exploit open-system design principles and architectures. This raises concerns about how to practically achieve dependability in software-intensive systems with many possible configurations when: 1) the actual configuration of the system is subject to frequent and possibly rapid change, and 2) the environment of typical reusable subsystems is variable and unpredictable. Our preliminary investigations indicate that current methods for achieving dependability in open architectures are insufficient. Conventional methods for testing are suited for stovepipe systems and depend strongly on the assumptions that the environment of a typical system is fixed and known in detail to the quality-assurance team at test and evaluation time. This paper outlines new approaches to quality assurance and testing that are better suited for providing affordable reliability in open architectures, and explains some of the additional technical features that an Open Architecture must have in order to become a Dependable Open Architecture.Naval Postgraduate School Acquisition Research ProgramApproved for public release; distribution is unlimited

Understanding, Assessing, and Mitigating Safety Risks in Artificial Intelligence Systems

Prepared for: Naval Air Warfare Development Center (NAVAIR)Traditional software safety techniques rely on validating software against a deductively defined specification of how the software should behave in particular situations. In the case of AI systems, specifications are often implicit or inductively defined. Data-driven methods are subject to sampling error since practical datasets cannot provide exhaustive coverage of all possible events in a real physical environment. Traditional software verification and validation approaches may not apply directly to these novel systems, complicating the operation of systems safety analysis (such as implemented in MIL-STD 882). However, AI offers advanced capabilities, and it is desirable to ensure the safety of systems that rely on these capabilities. When AI tech is deployed in a weapon system, robot, or planning system, unwanted events are possible. Several techniques can support the evaluation process for understanding the nature and likelihood of unwanted events in AI systems and making risk decisions on naval employment. This research considers the state of the art, evaluating which ones are most likely to be employable, usable, and correct. Techniques include software analysis, simulation environments, and mathematical determinations.Naval Air Warfare Development CenterNaval Postgraduate School, Naval Research Program (PE 0605853N/2098)Approved for public release. Distribution is unlimite

Applications and Meaning of Inheritance in Software Specifications

We present a novel inheritance mechanism for a specification language. This mechanism supports stepwise refinement by combining constraints that can be inherited from several sources. Inheritance in specifications differs from inheritance in programming languages. The proposed mechanism has been designed specifically to support computer-aided requirements analysis. The main design issues for the mechanism are explained, and the application of the mechanism to requirements analysis is illustrated via examples.This research was supported in part by the Army Research Office under grant number ARO-145-91. This research was supported in part by the National Science Foundation under grant number CCR-9058453

Weapon Systems Safety When Deploying AI Technology

NPS NRP Project PosterWe will investigate how AI technology that impacts system safety concerns should be developed. Guidance is needed for understanding likelihood of unwanted events and making risk decisions on naval employment of systems that incorporate such technology.Naval Air Warfare Center Weapons Division (NAWCWD)ASN(RDA) - Research, Development, and AcquisitionThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Weapon Systems Safety When Deploying AI Technology

NPS NRP Executive SummaryWe will investigate how AI technology that impacts system safety concerns should be developed. Guidance is needed for understanding likelihood of unwanted events and making risk decisions on naval employment of systems that incorporate such technology.Naval Air Warfare Center Weapons Division (NAWCWD)ASN(RDA) - Research, Development, and AcquisitionThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Generalization Per Category: Theory And Application

The concept of Generalization Per Category (GPO is formalized. It is shown that GPC imposes lattice structures on entity types and their subtypes. A high level application oriented data definition language based on the GPC is outlined which allows the system to derive general entity types and organize their instances. Users are freed from undue efforts in the design of databases which are about entity types with rich varieties and high populations. Effective browsing of these databases and efficient execution of frequent queries against them are achieved by using the lattice structures among the entity types and their subtypes

JAVA Wrappers for Automated Interoperability

Databases in Networked Information Systems International Workshop DNIS 2000This paper concentrates on the issues related to implementation of interoperability between distributed subsystems, particularly in the context of reengineering and integration of several centralized legacy systems. Currently, most interoperability techniques require the data or services to be tightly coupled to a particular server. Furthermore, as most programmers are trained in designing stand- alone application, developing distributed system proves to be time-consuming and difficult. Here, we addressed those concerns by creating an interface wrapper model that allows developers to treat distributed objects as local objects. A tool that automatically generates the features of Java interface wrapper from a specification language called the Prototyping System Description Language has been developed based on the model.U.S. Army Research OfficeARO 40473.30-MA-S

Object-oriented rapid prototyping

Object-oriented techniques form a promising approach for realizing an integrated computer-aided prototyping environment capable of detecting and correcting errors early in the development process. We discuss the connection between rapid prototyping, object-oriented data models, formal specifications, reusable components, and engineering databaseshttp://archive.org/details/objectorientedra00berzO&MN, Direct fundin

Which Unchanged Components to Retest after a Technology Upgrade

Proceedings Paper (for Acquisition Research Program)The Navy''s open architecture framework is intended to promote reuse and reduce costs. This paper focuses on exploiting open architecture principles to reduce testing effort and costs in cases in which the requirements and code for a subsystem have not been changed, but the code is running on new hardware and/or new operating systems due to a technology-advancement upgrade. This situation is common in Navy and DoD contexts such as submarine, aircraft carrier, and airframe systems, and accounts for a substantial fraction of the testing effort. Unmodified software components need to be retested after a technology upgrade in some, but not necessarily in all cases. This paper reports some early research on conditions under which testing of unmodified components can be avoided after a technology upgrade, outlines an approach for identifying situations in which retesting can be safely reduced, and indicates how to focus retesting in cases in which it cannot be avoided.Naval Postgraduate School Acquisition Research ProgramApproved for public release; distribution is unlimited